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Abstract 

Problem-based Learning (PBL) approach has been widely used in various disciplines since it is claimed to 
improve students’ soft skills. However, empirical supports on the effect of PBL on problem solving skills have 
been lacking and anecdotal in nature. This study aimed to determine the effect of PBL approach on students’ 
problem solving skills using a quasi-experimental non-equivalent group pretest-posttest design. Fifty 
management students from a premier Technical University in Malaysia were assigned to experimental and 
control groups. In the experimental group, students were given four problems to be solved and their solutions of 
the problems given were assessed in terms of their accuracy and quality. Students in the control group received 
conventional classroom instructional design. Results indicate that students in the experimental group have better 
problem solving skills (z: -4.220, p: 0.001 for accuracy and z: -2.594, p: 0.009 for quality) compared to those 
who were not exposed to the PBL approach. This finding substantiates the use of PBL as an effective 
instructional tool to improve students’ problem solving abilities. 
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1. Introduction 

A survey conducted by ManpowerGroup (2012) found that many employers are not satisfied with their current 
employees’ problem-solving skills. Similarly, fifty-five percent of employers participated in the survey by Grant 
Thornton LLP (AccountingWeb, 2010) claimed that recruiting accounting executives with necessary soft skills 
such as communication, critical thinking, and problem solving abilities poses the most significant challenge. The 
centrality of problem-solving skill is accentuated when it is found to enhance creativity, enjoyment, interest and 
learning (Yu et al., 2010). This issue has led to further scrutiny on instructional delivery at tertiary education 
level which mostly uses conventional classroom approach. A meta-analyses comparing PBL to conventional 
classrooms indicate that PBL approach is superior in terms of long-term retention, skill development and 
students satisfaction (Strobel & Barneveld, 2009) while Yadav et al. (2011) found that PBL leads to higher 
problem-solving skills and increase students’ factual knowledge (Yadav et al., 2011). Moreover, various studies 
in diverse disciplines such as Mathematics (Rohani & Sahar, 2012), Communication (Chapman, 2002), Medical 
(Nan, 2011), and Social Sciences (Nair & Alkiyumi, 2011) have shown that PBL have positive effects on 
students in building up their problem solving skills, critical and creative thinking skills, cooperative and 
communication skills, and adaptability (Albanese & Mitchell, 1993; Hmelo-Silver, 1998; Koray et al., 2008; and 
Nur Izzati et al., 2010). However, most of these studies are cross sectional in nature which leads to heated 
debates on its effectiveness. Anecdotal evidence in forms of inconsistent and contradicting findings also 
indicates the need to further investigate the effectiveness of PBL on problem solving skills. 

Thus, the main purpose of this study is to identify whether the students exposed to PBL treatments improve their 
problem solving skills. All students in PBL and control groups were given similar test at the beginning of the 
semester to establish the baseline score. The PBL group experienced the treatments while the control group was 
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having non-PBL approach in their learning process. At the end of the semester, another test was administered to 
all the students in both groups to gauge their problem solving skills. 

2. Method 

2.1 Design 

This was a quasi-experimental study with a non-equivalent control group, using pre and post analysis on 
measuring students’ problem solving skills in PBL learning environment and non-PBL learning environment. 
This design is appropriate for groups that are naturally assembled, such as intact classrooms (Campbell & 
Stanley, 1963). The design included the pre-test and post- test for both groups. The effect of instructional 
approach of learning (PBL) on students’ problem solving skills was measured in terms of the quality and 
accuracy of solutions provided by the students. 

2.2 Sample 

Both experimental and control groups had 25 participants in each group. All of these participants were first year 
undergraduate management students at Universiti Tun Hussein Onn Malaysia, enrolling in “Effective 
Communication” subject. The study was conducted for 14 weeks or one semester. Prior to exposure, all 
participants were given a test measuring their problem solving skills based on accuracy and quality of solutions. 
Later on, participants from PBL group were exposed to the PBL approach such as how to deal with the problems, 
preparing the “FILA” table-a four-column table (Facts, Ideas, Learning Issues, Action), and how to organize the 
information in solving the problems. Similar test was given at the end of the semester to all participants in both 
groups. Marks for pre and post measures tests were compared in indentifying whether PBL helps students in 
fostering students’ problem solving skills. 

2.3 Instruments 

The tests were prepared by the researchers and were verified by experts in content area, teaching and learning 
and PBL. The experts evaluated the content of the tests based on literature on crafting a good trigger, especially 
characteristics highlighted by Tan (2007). In getting the scores, the students’ answers were given to two raters. 
They assessed the students’ work on the accuracy and quality of the solutions. The accuracy and quality of the 
solutions were evaluated using the rubric provided by the researcher. The rubric was the modified version of the 
standardized MUET band description used in MUET examination. The scores were recorded and the data were 
analysed using Mann-Whitney U test. This test was selected in comparing the samples’ performances in 
assessing whether there is any significance difference in students’ problem solving skills from these two different 
groups at the beginning and at the end of the course. Furthermore, the number of respondent is 25 for each 
experimental and control groups, thus it can be concluded that this test is the most appropriate test to be used. 

Two facilitators gave their ratings on the students’ work. Marks were given and the average marks were recorded. 
Inter-raters of this study were chosen based on several criteria. They were experts in the course, shared the same 
interest, and had been teaching the same subject for several semesters. Initially three lecturers were chosen and 
they were given 10 scripts of the students’ work to be assessed. Based on the inter-rater agreement (kappa) result, 
two lecturers were chosen to be the inter-raters of this study. 


Table 1. Inter-rater agreement (Kappa) 



Kappa value 

P 

IR 1 and IR 2 

0.375 

0.03 

IR 1 and IR 3 

0.035 

0.775 

IR 2 and IR 3 

0.205 

0.159 


Kappa for the agreement between the ratings of inter-rater 1 and inter-rater 2 was 0.375 with p = 0.03. However, 
the agreement between the ratings of inter-rater 1 and inter-rater 3, followed by inter-rater 2 and inter-rater 3, 
were not significant with ^-values 0.775 and 0.159, respectively. Therefore, inter-rater 1 and inter-rater 2 were 
chosen to assess the students’ problem solving skills. 

2.4 Data Collection and Procedure 

At the beginning of the semester, the pre-test picture was conducted at the same time among the two classes 
identified. The subjects were given a problem in the form of a picture to measure their ability in solving the 


167 




www.ccsenet.org/ies 


International Education Studies 


Vol. 9, No. 5; 2016 


problem. The picture portrayed a clogged drain. 15 minutes were allocated for them to give solutions for the 
problem given. The test was conducted at the same time for PBL and non-PBL classes. 

A post-test picture was conducted at the end of the semester, after the subjects were exposed to PBL treatments. 
The post-test was a picture, which was similar to the one given at the beginning of the semester. The PBL and 
non-PBL classes sat for the test at the same time. The reason for having the post-test was to see if there was any 
difference in their problem solving skills before and after the exposure of the treatments. 

The scores on problem solving skills were processed using Statistical Package for Social Science (SPSS). All 
hypotheses were tested using non-parametric of Mann-Whitney U test. The non-parametric tests were conducted 
because the number of respondent was small (N = 25 for each PBL and non-PBL class). According to Siegel 
(1956), if the sample sizes are small, as small as N=6, non-parametric tests should be used unless the nature of 
the population distribution is known exactly. In addition, the adequate sample size for parametric tests is N>30 
(Tomkins, 2006). Non-parametric tests are also available for all common experimental designs (Siegel, 1956). In 
addition, non-parametric tests are suitable for treating samples made up of observations from several different 
populations (Siegel & Castellan, 1988). 

3. Results 

The objective of this study was to determine if there was any difference in students’ problem solving skills at the 
beginning and at the end of the semester after attending PBL treatments. 


Table 2. Participants’ demographic background (n=50) 


Characteristics 

PBL (n=25) Non-PBL (n=25) 

Gender: 

Male 

10 (20%) 

11 (22%) 

Female 

15 (30%) 

14 (28%) 

Age: 

20 

z: -.284,;? = 0.777 

4 

6 

21 

18 

12 

22 

3 

7 

Race: 

Malay 

z: -1.409,^ = 0.159 

9 (18%) 

11 (22%) 

Chinese 

13 (26%) 

10 (20%) 

Indian 

2 (4%) 

1 (2%) 

Others 

1 (2%) 

3 (6%) 


z: -.244,^ = 0.807 



168 




www.ccsenet.org/ies 


International Education Studies 


Vol. 9, No. 5; 2016 


Table 3. Pre and post measures on students’ problem solving skills between PBL and non-PBL (n=50) 


Non-PBL (n=25) 

PBL (n=25) 

Mean rank (accuracy post) 

15.50 

35.50 

Mean rank (quality post) 

20.88 

30.12 

Mann-Whitney U (accuracy) 


62.50 

(quality) 

197.00 


z(accuracy) 

-4.96 


z (quality) 

-2.913 


p (accuracy) 

0.001 


p (quality) 

0.004 



Table 1 displays the mean ranks of accuracy and quality on problem solving skills between PBL and non-PBL 
groups. It can be seen that there was a significant difference on the mean ranks between PBL and non-PBL in 
terms of accuracy and quality of the solution. For the PBL group, the mean rank was 35.50 compared to 
non-PBL group, 15.50. These results support the contention that there was improvement on problem solving 
skills in terms of accuracy after the students attended PBL treatments. 

From the aspect of quality, the mean rank value presented was 30.12 for PBL group and for the non-PBL group, 
the value was 20.88. Thus, it can be concluded that students in PBL group had better quality in their solutions of 
the problems. 

Furthermore, the z value displayed was -4.96 for accuracy and -2.913 for quality. The p values for accuracy and 
quality were 0.001 and 0.004 respectively (p< 0.05). Therefore, the Mann-Whitney U test of the experiment and 
control groups revealed that there was a statistically significant difference between these two groups on the 
accuracy and quality with respect to problem solving skills after the treatments. 

The finding of this study was consistent with the studies done by Flmelo-Silver (1998), Koray et al. (2008), Neo 
and Neo (2005), and Nur Izzati et al. (2010) despite the different types of measurements used. This indicates the 
suitability of PBL approach to be used in various disciplines. 

4. Discussion 

This study adds to existing literatures on the benefits of PBL approach compared to the traditional classroom 
approach. The problem-solving process, which is central within a PBL environment, was guided by a framework 
where students learn to explore and analyse problems systematically (Dochy et al., 2005). This enables students 
to be proactive and expand their existing knowledge bases. Students also use various sources of information 
from their textbooks and others in informal discussions with peers (Major & Palmer, 2001) which enable them to 
integrate knowledge across disciplines and approaches. Other than that, PBL provides channels for students to 
experience authentic problems (Savery & Duffy, 1995). By having authentic problems to be solved, students are 
more involved in the learning process. Therefore, students are usually more innovative and creative in their 
problem solving approach. In contrast, in a non-PBL approach of learning, students tend to have the knowledge 
rather than to create the knowledge and emphasis is on the broad coverage of content areas through lectures 
(Beachey, 2007). Traditional learning leads students to be passive in class, to memorize, repeat, and rely on notes 
given by the lecturers. 

When students are given a trigger by the lecturer (known as facilitator), they need to discuss and find solutions 
for the trigger. In the process of finding the solutions, they go through the process of information gathering (such 
as doing library research and search information from the Internet), conduct meetings, interact with people, and 
prepare the presentation (O’Grady & Alwis, 2002). Therefore, with enough information and ideas being 
discussed, the possibility to reach accurate and with quality solutions are there (Jonassen, 1997). 

5. Recommendations 

Future research might need to focus on the implementation of PBL at different levels of students, such as 
primary school, secondary school and part-time students. In addition, a larger sample should be considered and 
the assessments should be done not only for one semester but it could be throughout the whole course. This study, 
however, has several limitations inherent to quasi experimental research design. The findings of the research 
cannot be generalized to other settings, since it was employed with small samples of students at a premier 


169 




www.ccsenet.org/ies 


International Education Studies 


Vol. 9, No. 5; 2016 


Malaysian Technical University for only one semester. Furthermore, this study measured problem solving using 

two dimensions only, namely accuracy and quality. Thus, future research might include more comprehensive 

conceptualization and measurement of problem solving. 
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